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Abstract 


In 1999, Agilent Technologies introduced the first optical position sensor for 
mouse technology in the world. Besides functioning as part of a pointing 
device in computing, an optical mouse sensor can be considered as a general 
optical position sensor. One major drawback with the majority of optical 
displacement measuring tools lies with their relative high cost. Due to the 
economics of large volume production, the cost of an optical mouse sensor is 
extremely low. 

The optical mouse sensor has been shown to be able to provide highly 
accurate measurements of slow displacements with a resolution of 0.0635 
mm. Its efficiency for vibratory displacements sensing is also investigated 
and the work concludes that the optical mouse technology currently avail- 
able is workable, but has a confined scope of low frequency and low amplitude 
application in vibration motion sensing. As an optical position sensor, the 
optical mouse sensor has found positions in many industry applications. The 
optical mouse sensors are experimentally characterized in view of their use 
as multiple degree of freedom sensors for industrial application. In mobile 
robot applications, the optical mouse sensors have been employed for odom- 
etry measurement. In medical field, three optical mouse sensors are used to 
measure the movements of a device for tracking minimally invasive surgical 
(MIS) instruments in training setup. In neuroscience research, it is employed 
by an inexpensive submillisecond system for walking measurements of small 


animals. In microscopy field, it has been used as a region-of-interest position 


recorder and used in a digital readout manometer. In chemical field, it has 
been employed to measure viscoelastic deformation. 

In this thesis, the optical mouse sensor is used for surface shape analyzing, 
non-contact tactile sensing and combined tactile sensing respectively. 

Firstly, the optical mouse sensor is employed for surface shape analyzing. 
Besides of the flexible use of the optical mouse sensor, the proposed device is 
characterized by a small sensing area derived from the use of a ballpoint pen. 
The minimum of detectable height variance is restricted by the size of the 
ballpoint and the resolution of the optical mouse sensor. Experiments show 
that the proposed device can record the shape of smooth surface digitally 
and evaluate the roughness of rough surface statistically. 

Secondly, the optical mouse sensor is employed for non-contact tactile 
sensing. It is an interesting idea to make an optical mouse sensor work in 
the manner of a human hand but in a non-contact way to ”feel” the object 
surface. For the experiments on rubber surfaces, a periodically-square-shaped 
rubber surface and a periodically-arc-shaped rubber surface are studied in 
the experiments. Experiments show that the shape information of the object 
surfaces and the translation direction of the object surfaces can be obtained at 
the same time. For experiments on whitepaper, the translation direction, the 
height and the category of an object surface can be sensed. Based on the data 
table obtained from the experiments, the translation direction is calculated 
and the nonlinearity of the translation direction sensing is studied. It can be 
concluded that the resolution of an optical mouse sensor is direction-related. 
The feasibility of surface height measurement is analyzed and the optical 


mouse sensor can be used for crack detection on a plane surface. Height 


II 


variation of 0.2mm can be detected. Besides, the feasibility of the optical 
mouse sensor for surface discrimination is proposed. 

Thirdly, the optical mouse sensor is employed for combined tactile sensing 
. Based on the experimental characteristics of the optical mouse sensor, with 
a step-by-step table-look-up method, a combined tactile sensing system is 
proposed. When an object surface is moved underneath the optical mouse 
sensor, the translation direction, the category and the height of the object 
surface can be obtained with the recorded output of the optical mouse sensor 
based on a predefined data table. 

There are following advantages for the optical mouse sensor. The sensor 
interface is ready for use. The output of the system is digital, without the 
need of any additional analog-to-digital component. The driver and software 
for the sensing system is easily available. The outputs of the sensor can be 
intuitively displayed on the display and recorded on the hard disk, which 
greatly improves the convenience of the system. Because of the large-scale 
manufacture of the optical mouse sensor, the sensing system is cost-effective, 


and thus is easily available and generalized. 
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Chapter 1 


Introduction 


1.1 Optical Mouse 


In computing, a mouse is a pointing device that functions by detecting two- 
dimensional motion relative to its supporting surface. Physically, a mouse 
consists of a small case, held under one of the user's hands, with one or more 
buttons. It sometimes features other elements, such as ” wheels”, which allow 
the user to perform various system-dependent operations, or extra buttons 
or features can add more control or dimensional input. The mouse's motion 
typically translates into the motion of a pointer on a display, which allows 
for fine control of a Graphical User Interface. 

The name mouse, originated at the Stanford Research Institute, derives 
from the resemblance of early models (which had a cord attached to the rear 
part of the device, suggesting the idea of a tail) to the common mouse. 

Today’s computer mouse comes in many shapes, and in a wide range 


of features, sizes and prices. The two main technologies driving this input 


device are optical and mechanical. The mechanical engine was the earlier 
system and was introduced in the 1960s. Optical technology was introduced 
in the 1980s but didn't gain much traction until the early part of 2000. 

Since then, with the massive adoption of computers for communication, 
data storage, and networking, the mouse's function as an input device has 
grown increasingly more criticalparticularly in accommodating the higher 
tracking speeds and better response required by demanding users. 

Fueled by this insatiable appetite for performance, the mouse has come a 
long way from its humble beginnings. From a single button, corded device, 
now there is a version for every type of user including cordless models, and 


even high performance models oriented toward gaming. 


1.1.1 The Birth of the Computer Mouse 


The mouse was invented by Douglas Engelbart and developed the first one 
with his Chief Engineer, Bill English, of Stanford Research Institute (now 
SRI international) in 1963. It was intended as an input device for Engel- 
bart's oNLine System (NLS). Their creation used two perpendicular wheels 
attached to potentiometers to track its movement along the horizontal and 
vertical axes. 

In 1971, Xerox Palo Alto Research Center (PARC) signed an agreement 
with SRI licensing Xerox to use the mouse. The Xerox PARC mouse replaced 
the external wheels with a single ball, which could rotate in any direction. 
The ball’s motion was detected using perpendicular wheels connected to elec- 


trical commutators to move the onscreen cursor. The first Alto mouse was 


operational in 1972. Today's mechanical mouse owes a great deal to this 
Xerox PARC design. 

A couple of reasons for the slow adoption of mice from the 1970s to the 
early 1980s were the limited number of personal computers in the market, 
as well as their high prices. The Xerox PARC mouse in those days would 
set a buyer back a whopping $400 and required an additional $300 computer 
interface. 

Another milestone in mechanical mouse history came through Apple. Un- 
like prior mouse designs that used electrical commutators, the Apple mouse 
used optical encoders along the ball's equator, 90 degrees apart. 

Microsoft made its debut in the mouse market in 1983. Known as the 
” Green Eye Mouse” , the main function of this mouse was to provide navi- 
gation support for the GUI context of Microsoft Word for MS-DOS v. 1.00. 
The mouse has two protruding green buttons, hence its ” green eye” nick- 
name, and was equipped with a 25-pin D-plug to attach to the serial port on 
the original PC and compatibles. On the underside it had three small steel 
balls that allowed it to glide over a surface and a large steel ball in the center 
to register its position. 

One of the earliest optical mouse design started with the Mouse Sys- 
tems model. Commercially available from 1982 to 1995, this mouse used a 
four-segment photodiode chip, and could only be used on a special mirrored 
surface having a grid of fine lines. Ultimately several versions were avail- 
able for Amiga computers, while other versions were packaged with a PS/2 
connector to hook-up with an IBM PC or compatible. 

Then in 1985, Xerox introduced the 6085 Star, which featured the first 


optical mouse that was not tied to a precision surface. Although it was 
supplied with a pad having a printed dot pattern, it would also operate on 
other surfaces that had high-contrast printing. It could not, however, work 
on ordinary random surfaces such as most mouse pads or tabletops. 

In 1999, Agilent Technologies introduced a revolutionary optical position 
sensor. This innovative sensor operates by actually ”taking a picture” of 
the surface on which it is navigating, and comparing images sequentially to 
detect the speed and direction of movement. The device is able to navigate 
on a wide variety of surfaces, freeing the user from the space limitation of 
any mouse pad. 

Enhancing the optical technology, in September 2004, Agilent (now Avago 
Technologies) revealed laser illumination and tracking technology. Providing 
more surface tracking power than LED-based optical mice, laserbased op- 
tical mice with Avago LaserStream technology can easily track on painted 
metal, translucent plastics, frosted glass and many other previously difficult 
to navigate surfaces. Avago's laser engines also have increased navigation 
performance compared to position sensors using LED illumination. Boasting 
some of the highest performance benchmarks, including motion velocity up 
to 45 inches per second, frame rates in excess of 7,000 frames per second 
and a high resolution of 2,000 counts-per-inch, mouse performance has been 


bumped up a couple of notches. 


1.2 Optical Mouse Sensor 


The mechanical computer mouse, Douglas Engelbart's invention of the 1960's, 
is starting to look like an endangered species. While users praise its ingenuity, 
many resent its unreliability and its need for frequent cleaning. 

Change is on the way. Thanks to advances in optical navigation, solid- 
state optical mice have become the new standard. These new mice never 
need cleaning, track precisely and work on nearly all surfaces. They may 
even reduce repetitive stress injury, and to most people they just plain feel 
better. 

Their ease of use belies their complexity. Hidden inside the sleek plastic 
case is a sophisticated design, combining the best of today’s electronic and 


optical technology. 


1.2.1 Inside An Optical Mouse 


If you took apart an optical mouse and looked inside, youd find a complete 
imaging system. The mouse is essentially a tiny, high-speed video camera 
and image processor. 

As shown in Fig.1.1, a light-emitting diode (LED) illuminates the sur- 
face underneath the mouse. The light from the LED reflects off microscopic 
textural features in the area. 

A plastic lens collects the reflected light and forms an image on a sensor. 
If you were to look at the image, it would be a black-and-white picture of 
a tiny section of the surface. The sensor continuously takes pictures as the 


mouse moves. The sensor takes pictures quickly1500 pictures (frames) per 


Sensor 


Lens 
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Figure 1.1: Optical mice illuminate an area of the work surface with an LED, 
to reveal a microscopic pattern of highlights and shadows. These patterns 
are reflected onto the navigation sensor, which takes pictures at a rate of 
1500 images per second or more. 


second or morefast enough so that sequential pictures overlap. The images 


are then sent to the optical navigation engine for processing. 


1.2.2 The Basics of Optical Navigation 


The optical navigation engine is the brain of an optical mouse sensor. It 
identifies texture and other features in the pictures and tracks their motion. 
Much of the same visual material can be recognized in both frames. The 
underlying technology is to determine optical flow. 

Fig.1.2 illustrates how this is done. Two images were captured sequen- 
tially as the mouse was panned to the right and upwards. Much of the 
same visual material can be recognized in both frames. Through a patented 
image-processing algorithm, the optical navigation engine identifies common 
features between these two frames and determines the distance between them. 
This information is then translated into X and Y coordinates to indicate 


mouse movement. 


(a) (b) 


Figure 1.2: Optical mice illuminate an area of the work surface with an LED, 
to reveal a microscopic pattern of highlights and shadows. These patterns 
are reflected onto the navigation sensor, which takes pictures at a rate of 
1500 images per second or more. 


1.2.3 Advantages of Optical Technology 


Unlike traditional mechanical mice, optical mice have no moving ball that can 
clog up with dust or dirt. Users no longer need to perform regular cleaning 
for accurate tracking. In addition, optical technology can work on many 
surfaces where ball mice have difficulty, including curved surfaces or soft 
fabric. Because of their distinct advantages, optical mice are fast becoming 


the standard in computer stores around the world. 


1.3 Optical Position Sensor 


A wide variety of displacement sensors have been developed based on the 
principles of capacitive, inductive, magnetic, ultrasonic and optical sens- 
ing. For displacement sensors based on the optical effect alone, designs 
have been reported employing the principles of triangulation [7, 19], inter- 
ferometry [39,41], moire [2,21], diffraction [8,35], time-of-flight [23], and 
speckle [27,42]. Some of these schemes are able to provide measurements 
with sub-micrometer accuracy. Nevertheless, not all applications require 
such high degrees of measurement accuracy. In building drift monitoring 
and large-strain viscoelastic measurements, for example, mensurations in the 
range of 0.1 mm are generally sufficient. One major drawback with the ma- 
jority of optical displacement measuring tools lies with their relative high 
cost. This arises primarily from the expensive components used in certain 
designs and the non-economics of scale associated with low volume manufac- 
turing of such systems. There is, hence, an incentive to source for alternative 


cost-effective optical sensors for displacement measurement; in particular, 


when the demands of accuracy are not very high. 

The optical mouse sensor is used to implement a non-mechanical tracking 
engine for computer mice. It is based on optical navigation technology, which 
measures changes in position by optically acquiring sequential surface images 
(frames) and mathematically determining the direction and magnitude of 


movement. 


1.4 Application of Optical Mouse Sensor 


Computer mouse device manufacturing plays an important role in computer 
peripheral industry. In 1999, Agilent Technologies introduced the first optical 
position sensor for mouse technology in the world. 

Besides functioning as part of a pointing device in computing, an optical 
mouse sensor can be considered as a general optical position sensor. One 
major drawback with the majority of optical displacement measuring tools 
lies with their relative high cost. Due to the economics of large volume 
production, the cost of an optical mouse sensor is extremely low. 

The use of the computer mouse for scientific work has been previously 
reported [10,33,34]. The majority of these involve studies on visual tracking 
and human motor control [1,6]. 

Optical sensors are used extensively for displacement measurement. A 
cost-effective optical displacement sensor will be invaluable in applications 
where very high resolutions are not required. The optical mouse has been 
investigated to determine its suitability for two-dimensional displacement 


measurement [28]. While the mouse worked only on objects with opaque 


surfaces, experiments conducted with a commercial unit with 0.0635 mm 
resolution showed that highly linear (average R?-value of 0.9914) and low 
error (mean square error (M.S.E.) value below 0.018 mm?) measurements 
could be attained [26]. On the flipside, the unit could only operate if placed 
at a distance no greater than 1.25 mm from the object surface. Overall, 
the optical mouse has been found to be a viable two-dimensional displace- 
ment sensor. Its efficacy was demonstrated in measuring the viscoelastic 
elongation of polyethylene. While the viscoelastic behavior of materials is 
well studied, it is still actively researched due to its importance in a wide 
spectrum of engineering applications [12,18]. Optical methods to determine 
the viscoelastic deformation of materials have often relied on expensive and 
complicated designs. 

The optical mouse has been previously shown to be able to provide highly 
accurate measurements of slow displacements with a commercial unit with 
0.0635 mm resolution. Its efficiency for vibratory displacements is also in- 
vestigated [30]. It was found that frequency and amplitude thresholds ex- 
isted beyond which the position-time distributions obtained would not permit 
analysis. From the analyzable distributions derived, frequency estimation 
errors below 15% in magnitude could only be comfortably achieved if the 
frequency was limited to 35 Hz and the amplitude limited to 0.4 mm, while 
amplitude estimation errors below 20% in magnitude could only be safely 
attained if the frequency was limited to 5 Hz and the amplitude limited to 
0.2 mm. It concludes that the optical mouse technology currently available 
is workable, but has a confined scope of low frequency and low amplitude 


application in vibration motion sensing. Nevertheless, this range still makes 
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it useful as a dynamic sensor in a reasonable number of applications where 
low cost is particularly required. 

The optical mouse is an extremely cost-effective displacement sensor. 
In [31] the author describes the use of an optical mouse in an experiment to 
record position as a function of time for a simple mechanical oscillator. The 
recorded data correspond closely to that expected for damped harmonic os- 
cillations. These measurements can be made without the use of cumbersome 
attachments. The optical mouse is also advantageous in obviating concern 
about the integrity of electrical contact points and the deleterious effects of 
wear and of dirt accumulation on moving parts. 

A simple demonstration of viscoelasticity can be carried out by attaching 
a weight to a polymer film and watching it extend over time. For accurate 
and quantifiable data on the deformation, an electronic displacement sensor 
should be incorporated. Most of such sensors are expensive. In [29], an 
optical mouse was demonstrated to provide accurate data at low cost. 

The original mouse only had two wheels attached to encoders to measure 
horizontal and vertical displacement whereas the simplest and effective way 
to measure the displacement and trajectory of a mobile robot is to use an 
encoder placed in the drive motors, [3, 17,24] in the active wheels or even 
in additional passive wheels. However, encoder measurements suffer form 
problems of accurate wheel diameter estimation, uncertainty in the determi- 
nation of the contact point with the floor, dirt accumulation and wheel wear 
and slippage. 

In mobile robot applications, these problems play an important role be- 


cause the goal is to measure the robot displacement with the maximum ac- 
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curacy. To solve these problems additional sensors, such as ultrasonic, laser 
measurement system and vision [5] and are widely used to extract and com- 
pare additional relative scenario information to correct the estimated robot 
position. 

The optical mouse is proposed for indoor odometry measurement in mo- 
bile robot applications [9, 13, 15, 16, 36, 37]. The optical mouse is a very 
low-cost sensor and has the advantage that the measured displacement is 
independent from the kinematics of the robot because the optical sensor 
uses external natural microscopic ground landmarks to obtain the effective 
relative displacement. The sensor is calibrated and evaluated for odometry 
measurement. It was found that the original conception inside a visual feed- 
back loop precludes its use as an isolated displacement sensor although its 
impressive speed and parts, with a CMOS camera and a digital signal pro- 
cessor embedded on the same chip, suggests that an improved design can be 
a good alternative for accurate mobile robot odometry measurement. 

A novel, four degrees of freedom, low-cost device for tracking minimally 
invasive surgical instruments (MIS) in training setups was developed [22]. 
This device consists of a gimbal mechanism with three optical computer 
mouse sensors. The gimbal guides the MIS instrument, while optical sensors 
measure the movements of the instrument. To test the feasibility of using 
optical mouse sensors to track MIS instruments, the accuracy of these sensors 
was tested depending on three conditions: distance between lens and object, 
velocity of movements, and surface characteristics. The results of this study 
were used for developing a prototypeTrEndo. Tests performed on TrEndo 


show that the smallest movement that can be recognized by sensors is 0.06 
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mm for translation and 1.27” for rotation of the MIS instrument around 
its axis. The smallest recognized angle for rotation around incision point 
is 0.23. The accuracy of TrEndo is higher than 95%, and therefore allows 
tracking the movements of the MIS instrument. 

Low-cost optical motion sensors commonly used in computer mice are ex- 
perimentally characterized in view of their use as multiple degree of freedom 
sensors for industrial applications [25]. Each non-contact optical sensor pro- 
vides two output channels giving x and y components of the device relative 
displacement. The experimental activity presented in the paper shows that 
limitations mainly arise from the device sensitivity to the reference surface 
texture and from the upper limited working speed. It is shown that, for an 
effective application, the surface over which the sensor is moving must be co- 
operative. In case of a typical commercial device working on a white surface, 
the linearity error evaluated over a single pass trajectory of 200 mm is about 
0.1% whilst the variability due to the reference surface pattern orientation is 
of 2%. 

The need for simultaneous measurement of multiple degree-of-freedom 
(DOF) motions can be found in numerous applications such as robotic as- 
sembly, precision machining, optical tracking, wrist actuators, and active 
joysticks. Conventional single-axis encoders, though capable of providing 
high-resolution (linear or angular) measurements, rely on mechanical linkages 
(that often introduce frictions, backlashes, and singularities) to constrain the 
device so that the three-DOF (3-DOF) motion can be deduced from the in- 
dividual orthogonal measurements. The authors present here a noncontact 


optical sensor for 3-DOF planar and spherical orientation measurements [20]. 
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The authors begin with the operational principle of a microscopic-surface- 
based optical sensor, that is optical mouse sensor. The design concept and 
theory of a dual-sensor system capable of measuring 3-DOF planar and spher- 
ical motions in real time are then presented. Along with a detailed analysis, 
the concept feasibility of two prototype 3-DOF dual-sensor systems for mea- 
suring the instantaneous center of rotation and the angular displacement of 
a moving surface is demonstrated experimentally. It is expected that the 
analysis will serve as a basis for optimizing key design parameters that could 
significantly influence the sensor performance. 

Stimuli from a broad spectrum of sensory modalities, including visual, 
auditory, thermal, and chemical, can elicit walking responses in animals, 
reflecting neural activity in sensorimotor pathways. The authors have devel- 
oped an integrated walking measurement system with sub-millisecond tem- 
poral accuracy capable of detecting position changes on the order of 0.1 
mm [11]. This tracking system provides the experimenter with a means by 
which to map out the response spectrum of a tethered animal to any num- 
ber of sensory inputs on time scales relevant to propagation in the nervous 
system. The data acquisition system consists of a modified optical computer 
mouse, a microcontroller with peripheral support circuitry, a binary stimulus 
sync line, and a serial (RS-232) data transfer interface. The entire system 
is constructed of relatively inexpensive components mostly converted from 
commercially available peripheral devices. The authors have acquired walk- 
ing data synchronized with auditory stimuli at rates in excess of 2100 samples 
per second while applying this system to the walking phonotactic response 


of the parasitic fly Ormia ochracea. 
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In optical microscopy, the microscopic features of interest typically have to 
be derived from regions that are spatially distributed over the sample. While 
the features to be analyzed may be minute, the regions from where they must 
be obtained from may be located quite far apart from one another. If the 
features are not distinct enough to allow easy visual discrimination, it would 
be tedious and time-consuming to attempt to recall and revisit these regions 
of interest. One method to overcome this difficulty would be to note and 
record the graduated markings on the microscope, or to use built-in position 
encoders. The former is tedious while the latter is generally expensive. A 
region-of-interest position-recording implement based on the use of an optical 
mouse is presented [32]. It is inexpensive and easily adaptable to the manual 
stage of any optical microscope. 

The manometer remains a useful pressure measuring instrument in the 
laboratory and in industry despite it being discovered centuries ago. One 
of the major limitations of this instrument lies with its inability to produce 
digital readouts for automated data acquisition. The authors demonstrate 
this ability via the incorporation of an optical mouse to sense liquid level 
movement. The approach is very easy to implement and inexpensive. It is 
also shown to be able to provide digital pressure measurements with good 
accuracy and repeatability [38]. 

In [4], the authors presents a novel sensor for classification of material type 
and its surface roughness. The sensor is developed by means of a lightweight 
plunger probe and an optical mouse. An experimental prototype was de- 
veloped which involves bouncing or hopping of the plunger based impact 


probe freely on the plain surface of an object under test. The time and 
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features of bouncing signal are related to the material type and its surface 
properties, and each material has a unique set of such properties. During 
the bouncing of the probe, a time varying signal is generated from optical 
mouse that is recorded in a data file on PC. Some dominant unique features 
are then extracted using digital signal processing tools to optimize neural 
network based classifier used with the sensor. The classifier is developed 
on the basis of application of supervised structures of neural networks. For 
this, an optimum multilayer perceptron neural network (MLP NN) model is 
designed to maximize accuracy under the constraints of minimum network 
dimension. The optimal parameters of MLP NN model based on various per- 
formance measures and classification accuracy on the testing datasets even 
after attempting different data partitions are determined. The classification 
accuracy of MLP NN is found reasonable consistently in respect of rigor- 
ous testing using different data partitions. The performance of the proposed 
MLP NN based classifier has also been compared with the statistical classifi- 
cation trees approach. It is seen that the former one clearly outperforms the 


statistical approach. 


1.5 Our Work 


Our work can be dived into 3 parts. 

In Chapter 3, the optical mouse sensor is employed for surface shape 
analyzing. Besides of the flexible use of the optical mouse sensor, the pro- 
posed device is characterized by a small sensing area derived from the use 


of a ballpoint pen. The minimum of detectable height variance is restricted 
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by the size of the ballpoint and the resolution of the optical mouse sensor. 
Experiments show that the proposed device can record the shape of smooth 
surface digitally and evaluate the roughness of rough surface statistically. 

In Chapter 4, the optical mouse sensor is employed for non-contact tactile 
sensing. It is an interesting idea to make an optical mouse sensor work in 
the manner of a human hand but in a non-contact way to ”feel” the object 
surface. For the experiments on rubber surfaces, a periodically-square-shaped 
rubber surface and a periodically-arc-shaped rubber surface are studied in 
the experiments. Experiments show that the shape information of the object 
surfaces and the translation direction of the object surfaces can be obtained at 
the same time. For experiments on whitepaper, the translation direction, the 
height and the category of an object surface can be sensed. Based on the data 
table obtained from the experiments, the translation direction is calculated 
and the nonlinearity of the translation direction sensing is studied. It can be 
concluded that the resolution of an optical mouse sensor is direction-related. 
The feasibility of surface height measurement is analyzed and the optical 
mouse sensor can be used for crack detection on a plane surface. Height 
variation of 0.2 mm can be detected. Besides, the feasibility of the optical 
mouse sensor for surface discrimination is proposed. 

In Chapter 5, the optical mouse sensor is employed for combined tactile 
sensing . Based on the experimental characteristics of the optical mouse 
sensor, with a step-by-step table-look-up method, a combined tactile sensing 
system is proposed. When an object surface is moved underneath the optical 
mouse sensor, the translation direction, the category and the height of the 


object surface can be obtained with the recorded output of the optical mouse 
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sensor based on a predefined data table. 
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Chapter 2 


Technical Introduction 


This chapter introduces the hardware and software tool employed in the 
development of the thesis. These tools lay a solid foundation for the whole 


research work. 


2.1 Robot Arm 


In our study, to make the sensor move in a uniform speed, a robot arm is 
used to control movement of the optical mouse sensor, as shown in Fig. 2.1. 

A robot arm of model VS-6354DM is controlled by a robot controller of 
model RC5-VS6A. In each of X, Y and Z directions, the position repeatability 


of the robot arm of model VS-6354DM is +0.02mm. 


2.2 MATLAB and Pointer Tracking 


One of basic task of our study is to programm to track the mouse pointer on 


the screen. 
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Figure 2.1: A robot arm is used to control movement of the optical mouse 
sensor. 


MATLAB is a high-level language and interactive environment that en- 
ables you to perform computationally intensive tasks faster than with tradi- 
tional programming languages such as C, C++, and Fortran. 

In graphics, MATLAB supports Handle Graphics. Graphics objects are 
the basic elements used to display graphics and user interface elements. One 
of the graphics objects is root, that is top of the hierarchy corresponding to 
the computer screen. 

The main property of the root we used in the programm to track the 


pointer is PointerLocation. 
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[x,y] 

Current location of pointer. A vector containing the x- and y-coordinates 
of the pointer position, measured from the lower left corner of the screen. 
You can move the pointer by changing the values of this property. The Units 
property determines the units of this measurement. 

This property always contains the current pointer location, even if the 
pointer is not in a MATLAB window. A callback routine querying the Point- 
erLocation can get a value different from the location of the pointer when 
the callback was triggered. This difference results from delays in callback 
execution caused by competition for system resources. 

The flowchart of the MATLAB application for data recording is shown in 
Fig. 2.2. After the robot arm is started, a short delay is necessary to make 
the robot arm achieve the status of uniform motion. 


An instance of the experiments is as follows: 
1. Start the MATLAB application; 
2. Start the robot arm VS-6354DM; 


3. After a short acceleration, the optical mouse sensor reached the uniform 
motion stage, and the recording is started by the MATLAB application 


automatically; 
4. The obtained data are stored in the hard disk for further analysis. 


The output of the sensing system is a sequence (xi, yi), which describes 
the track of the pointer on the computer screen. Index i is integers in the 


interval [1, 1000], x is in the interval [1, 1024] and y is in the interval [1, 768]. 
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Figure 2.2: Flowchart of MATLAB application.After the robot arm is started, 
a short delay is necessary to make the robot arm achieve the status of uniform 
motion. 
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Chapter 3 


Surface Shape Analyzing 
Device Using Optical Mouse 


Sensor 


3.1 Introduction 


The main objective of the study is to propose a device to analyze the surface 
shape with flexible use of an optical mouse sensor. 

Instead of functioning as part of a pointing device in computing, an optical 
mouse sensor is used in the proposed device for surface shape analyzing. 
Besides of the flexible use of the optical mouse sensor, the proposed device is 
characterized by a small sensing area derived from the use of a ballpoint pen. 
The minimum of detectable height variance is restricted by the size of the 
ballpoint and the resolution of the optical mouse sensor. Experiments show 


that the proposed device can record the shape of smooth surface digitally 
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and evaluate the roughness of rough surface statistically. 


3.2 Sandpaper 


Sandpaper is a form of paper where an abrasive material has been fixed to 
its surface. 

Sandpaper is part of the ” Coated abrasives” family of abrasive products. 
It is used to remove small amounts of material from surfaces, either to make 
them smoother (painting and wood finishing), to remove a layer of mate- 
rial (e.g. old paint), or sometimes to make the surface rougher (e.g. as a 


preparation to gluing). 


3.2.1 Grit Sizes 


Grit size refers to the size of the particles of abrading materials embedded 
in the sandpaper. A number of different standards have been established 
for grit size. These standards establish not only the average grit size, but 
also the allowable variation from the average. The two most common are 
the United States CAMI (Coated Abrasive Manufacturers Institute, now 
part of the Unified Abrasives Manufacturers” Association) and the European 
FEPA (Federation of European Producers of Abrasives) ”P” grade. The 
FEPA system is the same as the ISO 6344 standard. Other systems used 
in sandpaper include the Japan Industrial Standards Committee (JIS), the 
micron grade (generally used for very fine grits). The ”ought” system was 
used in the past in the United States. Also, cheaper sandpapers sometimes 


are sold with nomenclature such as ” Coarse”, ” Medium” and ”Fine”, but it 
y) ) 
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is not clear to what standards these names refer. 


3.3 Device Structure 


The structure of the proposed device is illustrated in Fig.3.1. The heart of 
the device is an optical mouse sensor ADNS-2051. The ADNS-2051 can de- 
termine the direction and magnitude of movement at the maximum of 800 
dots per inch (dpi) and at speeds up to 14 inches per second. The ADNS- 
2051 contains an Image Acquisition System (IAS), a Digital Signal Processor 
(DSP), a two-channel quadrature output, and a two-wire serial port. The IAS 
acquires microscopic surface images via the lens and illumination system pro- 
vided by the HDNS-2100 lens, HDNS-2200 clip, and HLMP-ED80-XXXXKX 
LED. These images are processed by the DSP to determine the direction and 
distance of motion. The DSP generates the Ax and Ay relative displacement 
values that are converted into two channel quadrature signals. 

The exploded view drawing of the above-mentioned optical mouse com- 
ponents is illustrated in Fig. 3.2. The HDNS-2200 clip provides optical 
and mechanical coupling of the LED to the HDNS-2100 lens. The clip also 
provides a mechanical compression feature which locates over the ADNS- 
2051 sensor and is used to ensure that a light mechanical contact is always 
maintained between the ADNS-2051 sensor and the HDNS-2100 lens when 
assembled. The PCB provides the interface circuit between the ADNS-2051 
sensor and the computer to transmit the output of the ADNS-2051 sensor to 
the computer. The base plate is embedded into the housing of the device. 


In the housing, there is a slipper block. The slipper block is wooden and 
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Figure 3.1: Structure of surface shape analyzing device using optical mouse 
sensor 


therefore of less inertia. Compared with the wooden surface, the white paper 
surface is a more responsive work surface for the optical mouse sensor, and 
therefore a piece of white paper as the work surface is attached to the wooden 
surface next to the optical mouse sensor. 

Specially, at the bottom of the slipper block, part of a ballpoint pen is used 
as the ” antenna”. The structure of the ballpoint pen is just right a nice and 
low-cost mechanical structure to keep a smooth mechanical contact between 
the "antenna” and the object surface. Besides, at the top of the slipper 
block, a spring is attached to keep a durative mechanical contact between 


the ballpoint and the object surface. When the device sweeps the object 
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Figure 3.2: Exploded view drawing of optical mouse components 


surface, the optical mouse sensor stays fixed while the slipper block moves 
up and down responding to the height variances of the object surface. The 
maximum applicable surface height variance measurement range is restricted 
by the working range of the spring, and here it is about 15mm. The minimum 
of detectable height variance is restricted by the size of the ballpoint and the 
resolution of the optical mouse sensor. The smaller the size of the ballpoint 
and the higher the resolution of the optical mouse sensor, the smaller the 


minimum of detectable height variance. 


ZF 


3.4 Experiments 


The experiment system shown in Fig. 3.3 is set up to test the proposed device. 
A robot arm of model VS-6354DM is controlled by a robot controller of model 
RC5-VS6A. In each of X, Y and Z directions, the position repeatability of 
the robot arm of model VS-6354DM is +0.02mm. The device is held by the 
robot arm and is connected to a computer through a USB cable. On the 
computer, a MATALB program is developed to record the position of the 
pointer on the screen. The test object surface is fixed on the platform. 

A typical experiment instance is as follows: the proposed device held by 
the robot arm sweeps the object surface horizontally at a speed of 1mm/s 
, while the MATLAB application records the pointer position on the screen 
with a sampling interval of 0.1s. 

Before comprehensive tests on the device, the relative position between 
the robot arm and the platform are adjusted until the recorded data respond- 
ing to a smooth flat glass surface as reference surface are constant in the y 
axis on the screen during the experiment sampling time-interval. 

Three kinds of object surface are prepared to test the proposed device. 
It is the relative displacement of the pointer on the screen that makes sense. 
In order to unify the data processing, the first data point of each recorded 
data sequence is considered as zero displacement. 

Firstly, a smooth slope glass surface shown in Fig. 3.4 (a) is selected as a 
typical flat surface. As shown in Fig. 3.4 (b), the device shows good linearity 
with R = 0.9985. 


Some adjacent sampling points keep the same value is because that the 
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Figure 3.3: Experiment system for surface shape analyzing device using op- 
tical mouse sensor 


height variance is less than the minimum of detectable height variance. 
Secondly, a smooth semi cylindrical acrylic surface shown in Fig. 3.5 (a) 


is selected as a typical curved surface. The recorded data are shown in Fig. 
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Figure 3.4: Experimental setup (a) and results (b) for smooth slope glass 
surface 


3.5 (b), suggesting that the recorded data can approximately express the 
shape of curved surface. 

Thirdly, sandpaper set MH913 of grit size #100, #320 and #1000 are 
selected as typical rough surfaces, as shown in Fig. 3.6 (a). The larger the 
grit size number, the smoother the sandpaper. For sandpaper of each grit 


size, the experiments are repeated 5 times. One instance of recorded data 
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Figure 3.5: Experimental setup (a) and results (b) for smooth semi cylindrical 
acrylic surface. The device is at a height of 20mm to make the height variance 
of the object surface is in the maximum applicable surface height variance 
measurement range. 


sequence responding to sandpaper of each grit size is shown in Fig. 3.6 (b) 
and the standard deviations of the all the 5 instances are shown in Table. 
3.1. Confined by the minimum of detectable height variance, the proposed 
device can't track the detail of the sandpaper surfaces, but can estimate the 


roughness statistically. And the sandpaper of grit size #1000 is too smooth 
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to cause surface height variance more than the minimum of detectable height 


variance of the device. 


Displacement of Pointer on Screen (Pixel) 


Figure 3.6: Experimental setup (a) and results (b) for sandpaper set MH913. 
Intuitively, the less the grit size number, the more dispersive the recorded 
data. 


3.5 Conclusion 


There are following advantages for the proposed device, and most of them 
are derived from the employment of the optical mouse sensor. The output 
of the system is digital, without the need of any additional analog-to-digital 


component. The interface is ready for use. The driver and software for the 
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Table 3.1: Standard deviations of experimental results for sandpaper set 
MH913 


1 2 3 4 5 
#100 2.6870 2.4126 2.5655 2.7367 2.7918 
#320 0.7508 0.7398 0.6893 0.7239 0.6987 
#1000 0.0000 0.1414 0.2020 0.0000 0.0000 


sensing system is easily available. The outputs of the device can be intu- 
itively displayed on the display and recorded on the hard disk, which greatly 
improves the convenience of the system. Because of the volume manufacture 
of the optical mouse sensor, the sensing system is cost-effective, and thus is 
easily available and generalized. 

In summary, a device for analyzing the surface shape is reported, tak- 
ing advantage of an optical mouse sensor. For smooth surface, the device 
can record the surface shape digitally, and for rough surface, the device can 
estimate the roughness in a statistical way. The resolution of the device is 
restricted by the resolution of the optical mouse sensor and the size of the 


ballpoint in the ” antenna”. 
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Chapter 4 


Non-contact Tactile Sensing 


with Optical Mouse Sensor 


4.1 Introduction 


This chapter reviews the tactile sensing and then propose the primary idea 
of the non-contact tactile sensing with optical mouse sensor. Experiments 


on two groups of test objects are performed. 


4.2 Tactile Sensing 


Touch and tactile sensor are devices which measures the parameters of a 
contact between the sensor and an object. This interaction obtained is con- 
fined to a small defined region. This contrasts with a force and torque sensor 
that measures the total forces being applied to an object. In the considera- 


tion of tactile and touch sensing, the following definitions are commonly used: 


34 


Touch Sensing This is the detection and measurement of a contact force 
at a defined point. A touch sensor can also be restricted to binary 


information, namely touch, and no touch. 


Tactile Sensing This is the detection and measurement of the spatial dis- 
tribution of forces perpendicular to a predetermined sensory area, and 
the subsequent interpretation of the spatial information. A tactile- 
sensing array can be considered to be a coordinated group of touch 


sensors. 


Slip This is the measurement and detection of the movement of an object 
relative to the sensor. This can be achieved either by a specially de- 
signed slip sensor or by the interpretation of the data from a touch 


sensor or a tactile array. 


Tactile sensors can be used to sense a diverse range of stimulus rang- 
ing from detecting the presence or absence of a grasped object to a 
complete tactile image. A tactile sensor consists of an array of touch 
sensitive sites, the sites may be capable of measuring more than one 
property. The contact forces measured by a sensor are able to con- 
vey a large amount of information about the state of a grip. Texture, 
slip, impact and other contact conditions generate force and position 
signatures, that can be used to identify the state of a manipulation. 
This information can be determined by examination of the frequency 


domain, and is fully discussed in the literature. 
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As there is no comprehensive theory available that defines the sensing 
requirements for a robotic system, much of the knowledge is drawn 
from investigation of human sensing, and the analysis of grasping and 
manipulation. Study of the human sense of touch suggests that creating 
a gripper incorporating tactile sensing requires a wide range of sensors 
to fully determine the state of a grip. The detailed specification of a 
touch sensor will be a function of the actual task as it is required to 
perform. Currently no general specification of a touch or tactile sensor 
exists the following can be used as an excellent basis for defining the 
desirable characteristics of a touch or tactile sensor suitable for the 


majority of industrial applications: 


1. A touch sensor should ideally be a single-point contact, through the 


2 is con- 


sensory area can be any size. In practice, an area of 1-2 mm 
sidered a satisfactory compromise between the difficulty of fabricating 
a sub-miniature sensing element and the coarseness of a large sensing 


element. 


2. The sensitivity of the touch sensor is dependent on a number of vari- 
ables determined by the sensor’s basic physical characteristic. In addi- 
tion the sensitivity may also be the application, in particular any phys- 
ical barrier between the sensor and the object. A sensitivity within the 
range 0.4 to 10N, together with an allowance for accidental mechanical 


overload, is considered satisfactory for most industrial applications. 


3. A minimum sensor bandwidth of 100 Hz. 


36 


4. The sensors characteristics must be stable and repeatable with low 
hysteresis. A linear response is not absolutely necessary, as information 
processing techniques can be used to compensate for any moderate non- 


linearities. 


5. As the touch sensor will be used in an industrial application, it will 


need to be robust and protected from environmental damage. 


6. Ifa tactile array is being considered, the majority of application can be 
undertaken by an array 10-20 sensors square, with a spatial resolution 


of 1-2 mm. 


4.2.1 Touch Sensor Technology 


Many physical principles have been exploited in the development of tactile 
sensors. As the technologies involved are very diverse, this chapter can only 
consider the generalities of the technology involved. In most cases, the devel- 
opments in tactile sensing technologies are application driven. It should be 
recognized that the operation of a touch or tactile sensor is very dependant 
on the material of the object being gripped.. The sensors discussed in this 
chapter are capable of working with rigid objects. However if non-rigid mate- 
rial is being handled problems may arise. Work has shown that conventional 


sensors can be modified to operate with non-rigid materials. 


Mechanically Based Sensors 


The simplest form of touch sensor is one where the applied force is applied to 


a conventional mechanical micro-switch to form a binary touch sensor. The 
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force required to operate the switch will be determined by the its actuating 
characteristics and any external constraints. Other approaches are based on a 
mechanical movement activating a secondary device such as a potentiometer 


or displacement transducer. 


Resistive Based Sensors 


The use of compliant materials that have a defined force-resistance char- 
acteristics have received considerable attention in touch and tactile sensor 
research. The basic principle of this type of sensor is the measurement of 
the resistance of a conductive elastomer or foam between two points. The 
majority of the sensors use an elastomer that consists of a carbon doped 


rubber. 


Force Sensing Resistor 


A force sensing resistor is a piezoresistivity conductive polymer, which changes 
resistance in a predictable manner following application of force to its sur- 
face. It is normally supplied as a polymer sheet which has had the sensing 
film applied by screen printing. The sensing film consists of both electrically 
conducting and non-conducting particles suspended in matrix. The particle 
sizes are of the order of fraction of microns, and are formulated to reduce the 
temperature dependence, improve mechanical properties and increase surface 
durability. Applying a force to the surface of a the sensing film causes parti- 
cles to touch the conducting electrodes, changing the resistance of the film. 
As with all resistive based sensors the force sensitive resistor requires a rel- 


atively simple interface and can operate satisfactorily in moderately hostile 
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environments. 


Capacitive Based Sensors 


A capacitive touch sensor relies on the applied force either changing the 
distance between the plates or the effective surface area of the capacitor. 
In such a sensor the two conductive plates of the sensor are separated by a 
dielectric medium, which is also used as the elastomer to give the sensor its 
force-to-capacitance characteristics. 

To maximize the change in capacitance as force is applied, it is preferable 
to use a high permittivity, dielectric in a coaxial capacitor design. . In this 
type of sensor, as the size is reduced to increase the spatial resolution, the 
sensors absolute capacitance will decrease. With the limitations imposed by 
the sensitivity of the measurement techniques, and the increasing domination 
of stray capacitance, there is an effective limit on the resolution of a capacitive 
array. The figure shows the cross section of the capacitive touch transducer in 
which the movement of a one set of the capacitors’ plates is used to resolve 
the displacement and hence applied force. The use of a highly dielectric 
polymer such as polyvinylidene fluoride maximizes the change capacitance. 
From an application viewpoint, the coaxial design is better as its capacitance 
will give a greater increase for an applied force than the parallel plate design. 

To measure the change in capacitance, a number of techniques can be, 
the most popular is based on the use of a precision current source. A second 
approach is to use the sensor as part of a tuned or L.C. circuit, and measure 
the frequency response. Significant problem with capacitive sensors can be 


caused if they are in close proximity with the end effectors or robots earthed 
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metal structures, this leads to stray capacitance. This can be minimized by 
good circuit layout and mechanical design of the touch sensor. It is possible 
to fabricate a parallel plate capacitor on a single silicon slice, this can give a 


very compact sensing device. 


Magnetic Based Sensor 


There are two approaches to the design of touch or tactile sensors based 
on magnetic transduction. Firstly, the movement of a small magnet by an 
applied force will cause the flux density at the point of measurement to 
change. The flux measurement can be made by either a Hall effect or a mag- 
netoresistive device. Second, the core of the transformer or inductor can be 
manufactured from a magnetoelastic material that will deform under pres- 
sure and cause the magnetic coupling between transformer windings, or a 
coils inductance to change. A magnetoresistive or magnetoelastic material is 
a material whose magnetic characteristics are modified when the material is 
subjected to changes in externally applied physical forces. The magnetore- 
strictive or magnetoelastic sensor has a number of advantages that include 
high sensitivity and dynamic range, no measurable mechanical hysteresis, a 
linear response, and physical robustness. 

If a very small permanent magnet is held above the detection device by a 
complaint medium, the change in flux caused by the magnet's movement due 
to an applied force can be detected and measured. The field intensity follows 
an inverse relationship, leading to a nonlinear response, which can be easily 
linearized by processing. A one-dimensional sensor where a row of twenty 


hall effect devices placed opposite a magnet has been constructed. A tactile 
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sensor using magnetoelastic material has been developed, where the material 
was bonded to a substrate, and then used as a core for an inductor. As the 
core is stressed, the materials susceptibility changed, which is measured as a 


change in the coils inductance. 


Optical Sensors 


The rapid expansion of optical technology in recent years has led to the 
development of a wide range of tactile sensors. The operating principles of 


optical-based sensors are well known and fall into two classes: 


1. Intrinsic, where the optical phase, intensity, or polarization of trans- 


mitted light are modulated without interrupting the optical path 


2. Extrinsic, where the physical stimulus interacts with the light external 


to the primary light path. 


Intrinsic and extrinsic optical sensors can be used for touch, torque, and force 
sensing. For industrial applications, the most suitable will be that which 
requires the least optical processing. For example the detection of phase 
shift, using interferometry, is not considered a practical option for robotic 
touch and force sensors. For robotic touch and force-sensing applications, the 
extrinsic sensor based on intensity measurement is the most widely used due 
to its simplicity of construction and the subsequent information processing. 
The potential benefits of using optical sensors can be summarized as follow: 

Immunity to external electromagnetic interference, which is widespread 


in robotic applications. 


1. Intrinsically safe. 


Al 


2. The use of optical fibre allows the sensor to be located some distance 


from the optical source and receiver. 


3. Low weight and volume. 


Touch and tactile optical sensors have been developed using a range of optical 
technologies:(a) Modulating the intensity of light by moving an obstruction 


into the light path.(b) Photoelasticity. 


Optical Fibre Based Sensors 


Tactile sensors can be constructed from the fibre itself. A number of tactile 
sensors have been developed using this approach. In the majority of cases 
either the sensor structure was too big to be attached to the fingers of robotic 
hand or the operation was too complex for use in the industrial environment. 
A suitable design can be based on internal-state microbending of optical 
fibres. Microbending is the process of light attenuation in the core of fibre 
when a mechanical bend or perturbation (of the order of few microns) is 
applied to the outer surface of the fibre. The degree of attenuation depends on 
the fibre parameters as well as radius of curvature and spatial wavelength of 
the bend. Research has demonstrate the feasibility of effecting microbending 
on an optical fibre by the application of a force to a second orthogonal optical 


fibre. 


Piezoelectric Sensors 


Polymeric materials that exhibit piezoelectric properties are suitable for use 


as a touch or tactile sensors, while quartz and some ceramics have piezoelec- 
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tric properties, polymers such as polyvinylidene fluoride (PVDF) are nor- 
mally used in sensors. 

Polyvinylidene fluoride is not piezoelectric in its raw state, but can be 
made piezoelectric by heating the PVDF within an electric field. Polyvinyli- 
dene fluoride is supplied sheets between as 5 microns and 2 mm thick, and 
has good mechanical properties. A thin layer of metalization is applied to 
both sides of the sheet to collect the charge and permit electrical connections 
being made. In addition it can be moulded, hence PVDF has number of 


attraction when considering tactile sensor material as an artificial skin. 


Strain Gauges in Tactile Sensors 


A strain gauge when attached to a surface will detect the change in length 
of the material as it is subjected to external forces. The strain gauge is 
manufactured from either resistive elements (foil, wire, or resistive ink) or 
from semiconducting material. A typical resistive gauge consists of the re- 
sistive grid being bonded to an epoxy backing film. If the strain gauge is 
pre-stressed prior to the application of the backing medium, it is possible to 
measure both tensile and compressive stresses. The semi-conducting strain 
gauge is fabricated from a suitable doped piece of silicone, in this case the 
mechanism used for the resistance change is the piezoresistive effect. 

When applied to robotic touch applications, the strain gauge is normally 
used in two configurations: as a load cell, where the stress is measured di- 
rectly at the point of contact, or with the strain gauge positioned within the 


structure of the end effector. 
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Silicon Based Sensors 


Technologies for micromachining sensors are currently being developed world- 
wide. The developments can be directly linked to the advanced processing 
capabilities of the integrated circuit industry, that has developed fabrica- 
tion techniques that allow the interfacing of the non-electronic environment 
to be integrated through micro-electromechanical systems.. Though not as 
dimensionally rigorous as the more mature silicon planer technology, micro- 
machining is inherently more complex as it is involves the manufacture of a 
three-dimensional object. Therefore the fabrication relies on additive layer 
techniques to produce the mechanical structure. 

The excellent characteristics of silicon, that has made micromachined 
sensors possible, include a tensile strength comparable to steel, elastic to 
breaking point, and there is very little mechanical hysteresis in devices made 
from as single crystal, a low thermal coefficient of expansion. 

To date it is apparent that microengineering has been applied most suc- 
cessfully to sensors. Some sensor applications take advantage of the device- 
to-device or batch-to-batch repeatability of wafer-scale processing to remove 
expensive calibration procedures. Current applications are restricted largely 
to pressure and acceleration sensors, though these in principle can be used 
as force sensors. As the structure is very delicate, there still are problems in 


developing a suitable tactile sensor for industrial applications. 
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Smart Sensors 


The most significant problem with the sensor systems discussed so far is that 
of signal processing. Researchers are therefore looking to develop a complete 
sensing system rather than individual sensors, together with individual inter- 
faces and interconnections. This allows the signal processing to be brought 
as close as possible to the sensor itself or integrated with the sensor. Such 
sensors are generally termed smart sensors. It is the advances in silicon 
fabrication techniques which have enabled the recent developments in smart 
sensors. There is no single definition of what a smart sensor should be capable 
of doing, mainly because interest in smart sensors is relatively new. However, 
there is a strong feeling that the minimum requirements are that the sensing 
system should be capable of self diagnostics, calibration and testing. As sil- 
icon can be machined to form moving parts such as diaphragms and beams, 
a tactile sensor can in principal be fabricated on single piece of silicon. Very 
little commercial success has been obtained so far, largely due to the prob- 
lems encountered in transferring the technology involved from the research 
laboratory to industry. In all tactile sensors their is a major problem of in- 
formation processing, and interconnection. An array has 2n connection and 
individual wires, any reduction in interconnection requirements is welcomed 
due to ease of construction and increased reliability. A number of researchers 
have been addressing the problem of integrating a tactile sensor with integral 
signal processing. In this design the sensors conductive elastomer sheet was 
placed over a substrate. The significant feature of this design is that the 


substrate incorporated VLSI circuitry so that each sensing element not only 
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measures its data but processes it as well. Each site performs the measure- 
ments and processing operations in parallel. The main difficulty with this 
approach was the poor discrimination, and susceptibility to physical damage 
However, the VLSI approach was demonstrated to be viable, and alleviated 


the problems of wiring up each site and processing the data serially. 


Multi-stimuli Touch Sensors 


It has been assumed that all the touch sensors discussed in this section re- 
spond only to a force stimulus. However, in practice most respond to other 
external stimuli, in particular, temperature. If PVDF has to be used as a 
force sensor in an environment with a widely varying ambient temperature, 
there may be a requirement for a piece of unstressed PVDF to act as a 
temperature reference. It is possible for a sensor to respond both to force 
and temperature changes. This has a particular use for object recognition 
between materials that have different thermal conductivity, e.g., between a 
metal and a polymer. If the complexity of the interpretation of data from 
PVDF is unsuitable for an application, touch sensors incorporating a resis- 
tive elastomer for force, and thermistors for temperature measurement can 
be constructed. By the use of two or more force-sensitive layers on the sensor, 
which have different characteristics (e.g., resistive elastomer and PVDF), it 


is possible to simulate the epidermal and dermal layers of human skin. 
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4.3 Primary Idea for Non-contact Tactile Sens- 
ing 


One of the important performance specifications of an optical mouse sensor 
is the resolution. Resolution reflects the number of steps the optical mouse 
sensor will report when it moves one inch, expressed in dpi (dots-per-inch). 
Practically, if an optical mouse sensor is in a uniform motion for a constant 
time, the displacement of the pointer on the display is proportional to the 
resolution of the optical mouse sensor, and thus the displacement of the 
pointer on the display can be employed as a measure of the resolution. 


Fig. 4.1 shows the distance from lens reference plane to surface, denoted 


as Z. 
Lens 
Sensor 
j Object surate N Lens Reference Plane 


Figure 4.1: Distance from lens reference plane to surface 


In recommended operating condition for the optical mouse sensor, the 
minimum of z is 2.3mm, the maximum of z is 2.5mm, and the typical of z is 
2.4mm. In normal cases, the object surface is flat, and the distance between 
the optical mouse sensor and the object surface is constant. 


Intuitively, during the daily use, if an optical mouse is lifted away from 
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the working surface and the height from the optical mouse to the working 
surface is above some value, the resolution of the optical mouse decreases 
to zero. It can be seen that the resolution of an optical mouse sensor is 
related to the height from the optical mouse to the working surface. It is 
an interesting idea to make an optical mouse sensor work in the manner of 
human hand but in a non-contact way to ”feel” the object surface based on 
the characteristics mentioned above. 

If the object surface is not flat, when the optical mouse sensor is moved 
horizontally, z is variable, a variable resolution will show, which will influence 
the pointer motion on the computer screen. So analyzing the pointer track on 
the screen can provide some information of the shape of the object surface. 

Based on the ideas mentioned above, in following experiments, firstly, two 
rubber surfaces with regular surface shapes are prepared, an optical mouse 
sensor which is placed over the object surfaces is controlled by a robot arm, 
and a MATLAB application is developed to record the pointer motion on the 
computer screen; and secondly, whitepaper is used as the test object in the 


same experiment procedures. 


4.4 Experiment Setup 


The diagram of the experiment system is shown in Fig. 4.2. 

The relative coordinate system is shown as Fig. 4.3. 

The x-y coordinate system is the coordinate system for the object surface 
and the a-b coordinate system is the coordinate system for the optical mouse 


sensor. Here, a is a variable angle between a-axis and x-axis. 
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Figure 4.2: Diagram of the experiment system for non-contact tactile sensing 


A robot arm of model VS-6354DM is controlled by a robot controller of 
model RC5-VS6A. Both of VS-6354DM and RC5-VS6A are manufactured 
by DENSO Corporation. In each of X, Y, and Z directions, the position 
repeatability of the robot arm of model VS-6354DM is +0.02 mm. 

The optical mouse sensor used for the experiment is from an optical mouse 
manufactured by Logitech, and the product number is SOM-U20. The part 
number of the optical mouse sensor is $2599, which is manufactured by Ag- 


ilent Technologies. The nominated resolution of $2599 is 400dpi. 
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Figure 4.3: Sensing coordinate systems for non-contact tactile sensing 


In the Windows XP operating system, the pointer speed is set to the 
fastest and the property ” Enhance pointer precision” is enabled, which help 
improve the system sensitivity, as shown in Fig. 4.4. 

The output of the sensing system is a sequence (x;, yi), which describes 
the track of the pointer on the computer screen. Index 7 is integers in the 


interval [1, 1000], x is in the interval [1, 1024] and y is in the interval [1, 768]. 
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Figure 4.4: System configuration of mouse in the experiments 
4.5 Experiment on Rubber Surfaces 


The sensing objects are two rubber surfaces. Two object surfaces with differ- 
ent surface shapes are prepared, that is the periodically-square-shaped rubber 
surface and the periodically-arc-shaped rubber surface. The 3-D view of the 
two object surfaces are shown in Fig. 4.5. 

In order to get the characteristics of resolution vs. height for the rubber 
surface used in the experiments, z is changed from 1.4mm to 2.8mm by 
a step of 0.2mm and a is changed from 0° to 90° by a step of 30°. The 
translation velocity of the robot arm is set as 1mm/s, and the translation 
time is 10s. For the uniform translation, the displacement of pointer on the 
screen is proportional to the resolution of the optical mouse sensor. The 
characteristics of resolution vs. height are shown in Fig. 4.6. It can be seen 
that the characteristics are monotonic in the interval [1.4mm, 2.8mm], and 


surface height variation of 0.2mm can be measured. 


4.5.1 Tests on the Two Rubber Surfaces 


In the tests, the sampling interval is 0.05s and each group of data includes 


1000 points, so the total sampling time for each group is 50s. 
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Figure 4.5: 3-D view of the two rubber surfaces used for non-contact tactile 
sensing 


Firstly, the periodically-square-shaped surface is placed under the optical 
mouse sensor, and o is set as 0°. Then the optical mouse sensor is moved 
over the object surface at a uniform speed of 1mm/s in the y-axis direction 
controlled by the robot arm VS-6354DM, while the MATLAB application 
records the pointer positions on the screen. The angle between a-axis and 
x-axis a is changed from 0° to 90° by a step value of 10°, and the above 
procedures are repeated. 


Secondly, procedures above are repeated on the periodically-arc-shaped 
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Figure 4.6: Characteristics of resolution vs. height for the rubber surface 


surface. 

Finally, 20 track curves of the pointer on the screen are obtained. 

In order to simplify the data processing, the starting point of each track 
curve coordinates is translated to (0, 0), because it is the relative displace- 
ment that makes sense. In order to simplify the explanation, 6 typical track 


curves are employed. 


4.5.2 Translation Direction Measurement 


In the experiments, the object surfaces are fixed, and the obtained direction 
is the direction of the translation of the optical mouse sensor, as shown in 
Fig. 4.7 and Fig. 4.8. 

The former is obtained when the object surface is periodically-square- 
shaped, and the later is obtained when the object surface is periodically-arc- 


shaped. 
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Figure 4.7: Track curves of different directions for periodically-square-shaped 
surface 
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Figure 4.8: Track curves of different directions for periodically-arc-shaped 
surface 


From the point of relative movement, the translation direction of the 


surface and the direction of the surface textures can be obtained, as shown 
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in Fig. 4.9 and Fig. 4.10. The slope of the linear part reveals the angle. 
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Figure 4.9: Different translation directions in non-contact tactile sensing 
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Figure 4.10: Different texture directions in non-contact tactile sensing 


During the experiment, it is found that the a-axis sensitivity and b-axis 
sensitivity of the optical mouse sensor are different. For example, in the case 
of a = 70°, in Fig. 4.11, if the a-axis sensitivity and b-axis sensitivity of the 


optical mouse sensor are the same, the track curve for œa = 70° and the curve 
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of y/x = arctan(70°) will be of the same slope. It can be seen that a-axis 
sensitivity is less than b-axis sensitivity. This is because the illumination of 


the LED is from the side, not vertically and symmetrically. 
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Figure 4.11: a-axis and b-axis of different resolutions (œ = 70°) 


4.5.3 The Surface Shape of the Object Surface 


The surface height is one kind of description of the surface shape. If the height 
variation range of the object surface is less than the range of the monotonic 
interval of the characteristics of resolution vs. height, by setting the initial 
distance between the lens reference plane and object surface properly, the 
height variation interval of the object surface can fall in the monotonic in- 
terval. When the optical mouse sensor is moved over the object surface at 
a uniform speed, for a fixed direction, the slope of the track curve of the 
pointer on the screen is a measure of the surface height. When the surface 


height is the same, the slope of the track curve is the same. In Fig. 4.7 and 
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Fig. 4.8, the periodically-square-shaped rubber surface and the periodically- 


arc-shaped rubber surface are used to illustrate the fact qualitatively. 


4.6 Experiments on Whitepaper 


4.6.1 Experiments Procedures 


Avago Technologies provides a computer application named ” Cursor Tester” 
for the mouse bench test. Before the experiments, ” Cursor Tester” is em- 
ployed for direction calibration and parallelism calibration. The initial posi- 
tion and initial pose of the optical mouse sensor are adjusted by the robot 
arm while the outputs of ” Cursor Tester” are observed and analyzed. After 
several recurrences of calibration, when a = 0° , if the optical mouse sensor is 
moved in the x-axis direction, there is zero displacement in the vertical axis 
direction on the display; if the optical mouse sensor is moved in the y-axis 
direction, there is zero displacement in the horizontal axis direction on the 
display; and the lens reference plane is parallel to the object surface. 

In the experiments, for a given z and a given a initially set by the robot 
arm, the optical mouse sensor is moved over the object surface at a uniform 
velocity of Imm /s and in the y-axis direction, the pointer position on the 
display varies and the pointer position is recorded by the MATLAB applica- 
tion. 

In the MATLAB application, the sampling interval is set as 0.05s and 
the sampling length is set as 100, so the total sampling time for each data 


group is 5s. The output of the sensing system is a sequence (2;,y;), which 
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describes the track of the pointer on the computer display and the length of 
the sequence is 100. x; is in the interval [1, 1024] and y; is in the interval [1, 
768]. 

In order to simplify data processing, the starting point of each pointer 
track curve coordinates is translated to (0, 0), because it is the relative 
displacement that makes sense. 

For RHQ-100, z = [1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0, 3.2] mm, and a = 
[0, 10, 20, 30, 40, 50, 60, 70, 80, 90] °. Finally, 90 sequences of dimension 


100 x 2 are obtained. 


4.6.2 Translation Direction Measurement 


The 1st dimension and 2nd dimension of the data table, that is (x;, y;), are 
employed to calculate œ. One estimate of tan() is the solution in the least 
squares sense to the overdetermined system of equations Y100x1 tan(a’) = 
X100x1. For calculation of each tan(a’), the average of 9 solutions of equations 
corresponding to 9 values of z are used. The relation between the estimated 
angle a’ and the actual angle a is shown in Fig. 4.12. It can be seen that a’ 
and a is not linearly related and a cubic polynomial provides a good fitting. 
That is, the angle a’ presented on the display is not linearly related to the 
angle a set by the robot arm. 

Furthermore, it can be concluded that the a-axis resolution and b-axis 
resolution of the optical mouse sensor are different. This is because the 
illumination of the LED is from the side, not vertically and symmetrically. 


Hereafter, the default resolution is referred to the b-axis resolution. 
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Figure 4.12: The relation between the estimated angle a’ and the actual 
angle a. 


4.6.3 Height Measurement 


Based on the data table with a = 0°, the characteristics of resolution vs. 
height of whitepaper RHQ-100 is obtained, as shown in Fig. 4.13. In the 
interval [1.6, 2.2] mm, the characteristics curve is monotonically increasing, 
and in the interval [2.2, 3.2] mm, the characteristics curve is monotonically 
decreasing. 

In the characteristics of resolution vs. height of an object surface, if 
the height variation range of the object surface is less than the range of a 
monotonic interval of the characteristics of resolution vs. height, by setting 
the initial height between the lens reference plane and the object surface 
properly, the height variation interval of the object surface can fall into the 
monotonic interval. In such a condition, the characteristics of resolution vs. 


height can be used for height measurement effectively. It can be seen that 
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Figure 4.13: The characteristics of resolution vs. height of whitepaper RHQ- 
100. 
height variation of 0.2mm can be detected. 

For different object surface, the position and range of the monotonic 


interval of the characteristics of resolution vs. height are different. 


4.6.4 Crack Detection 


Based the feasibility of height measurement of an optical mouse sensor, it 
can be used for crack detection on a plane surface. 

An object surface with specific crack configuration is constructed using 
whitepaper RHQ-100 as shown in Fig. 4.14. The zero reference height is 
z = 3.2 mm. The thickness of 2 pieces of whitepaper is about 0.2 mm. 
The thickness of the crackless object surface is of 4 pieces of whitepaper, 
the thickness of interval [5, 7.5] mm is of 2 pieces of whitepaper, and the 


thickness of interval [12.5, 17.5] mm is of 8 pieces of whitepaper. 
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Figure 4.14: An object surface with specific crack configuration. 


When a = 0°, the optical mouse sensor is moved over the constructed 
object surface at a uniform velocity of Imm/s, and the pointer position on 
the display is recorded by the MATLAB application with a sampling interval 
of 0.05s. The position of the pointer on the display varies with the position of 
the optical mouse sensor in the y-axis direction, as shown in Fig. 4.15 with 
piecewise linear fitting. The solid lines correspond to the crackless parts, 
while the dashed lines correspond to the cracked parts. 

A forward difference operation is performed on the curve in Fig. 4.15, 
and the result curve is shown in Fig. 4.16. 

When the optical mouse sensor is moved over the object surface at a 
uniform velocity, the slope of the displacement curve of the pointer on the 


display is a measure of the surface height. When the surface heights are the 
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Figure 4.15: The position of the pointer on the display varies with the posi- 
tion of the optical mouse sensor in the y-axis direction. 
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Figure 4.16: A forward difference operation is performed on the curve in Fig. 
4.15. 


same, the slopes of the corresponding displacement curve are the same. 
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4.6.5 Category Discrimination 


As a human hand touches different object surfaces, the human feel is slightly 
different, and an optical mouse sensor can also feel the differences in a non- 
contact way based on the characteristics of resolution vs. height of different 
object surfaces. In such a case, the sensed surface differences are mainly 
roughness differences. 

Fig. 4.17 is from a datasheet from Agilent Technologies. Burl Formica, 
white paper, manila, black copy and black walnut are standard surfaces which 
Agilent Technologies recommends to be used for surface navigation testing. 

If the height from lens reference plane to object surface is selected prop- 
erly, the surface discrimination can be performed based on the resolution 
differences at a specific height. 

For each object surface, the characteristics of resolution vs. height are 
specific, and a specific characteristics curve can be selected by a specific 
curve segment. If the height from lens reference plane to object surface is 
unknown, in order to recognize a specific characteristics curve, two optical 
mouse sensors can cooperate, with a calibrated height difference (e.g. 0.2 
mm) between the two optical mouse sensors. By matching the output of the 
two optical mouse sensors, a specific characteristics curve can be selected, 
and therefore the category of the object surface can be felt. 

The characteristic of resolution vs. height of white paper in Fig. 4.17 
differ much from that in Fig. 4.13. According to Application Note 5274 
” Optical Navigation - Standard Surface Kit” from Agilent, the part number 
of the white paper they tested is OCE 8650000042 different from RHQ-100. 
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Figure 4.17: Burl Formica, white paper, manila, black copy and black walnut 
are standard surfaces which Agilent Technologies recommends to be used for 
surface navigation testing. 


4.7 Discussion 


In the experiments above, x is in the interval [1, 1024] and y is in the interval 
[1, 768]. In actual applications, there may be some cases where x and y 
are beyond ranges above. In fact, this is a limitation caused by the driver 


software of the computer mouse. In actual applications, the direct output 
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signals of the optical mouse sensor, which is a two-channel quadrature output, 
can be used for processing. In such a way, without the intervention of the 
operating system and the driver, higher sensitivity and higher measurement 
speed can be expected. Furthermore, the two-channel quadrature output 
can be transformed by logic circuits to be used for PWM (Pulse Width 
Modulation) control in some industry applications. 

In the experiments above, the nominated resolution of the employed op- 
tical mouse sensor is 400dpi. In the mainstream market, the resolution of an 
optical mouse sensor is 800dpi or higher. If higher sensitivity is required, an 
optical mouse sensor of higher resolution can be employed. 

Position jitter is one of the noise sources of the measurement. Position 
jitter is an undesirable false movement of the pointer when an optical mouse 
is static. Such movements are usually small and can be seen on the display as 
jittering or even quivering. In our experiments, such an effect is superposed 
on the expected movement of the optical mouse sensor, and has been canceled 
by averaging and fitting. 

As for the application of crack detection, real-time on-line detection can 
be expected by developing a novel detection algorithm. 

In order to simplify the experiment setup and show the resolution of the 
system intuitively, we used the whitepaper as the test object. Practically, 
we can study the characteristics of resolution vs. height on each applicable 
surface to design a crack detection system for it. 

Besides the height from lens reference plane to object surface and the 
category of the object surface, many other factors, such as the velocity of 


the sensor, LED current level and the wavelength responsibility of the LED, 
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influence the resolution and the output of the optical mouse sensor. In our 
current research, these parameters are constant, and the effects of them will 
be studied in further research work. 

Besides the LED-based optical mouse sensor, the laser optical mouse 
sensor with superior performance is gradually popularized. The laser opti- 
cal mouse sensor is enabled with Agilent's LaserStream Technology. Laser- 
Stream navigation engines provide much finer precision and more accurate 
tracking on a wider variety of surfaces, part of which proves difficult for 
traditional LED-based optical navigation, such as glossy and reflective sur- 
faces. The adoption of the LaserStream Technology will help improve the 
performance and application of the proposed system. 

Furthermore, the essence of an optical mouse sensor is optical flow. For an 
optical mouse, the lens focuses the camera onto the surface directly below the 
mouse and thus an optical mouse only tracks the surface directly touching or 
very slightly above the surface it is tracking. By interfacing an optical mouse 
sensor to a microcontroller and replacing the default lens with a lens focusing 
at a distance away, it should be possible to harness this cheap optical flow 


technology for many more tasks. 


4.8 Conclusion 


Instead of as a traditional position sensor, an optical mouse sensor can be 
used for non-contact tactile sensing to feel the translation direction, the 
height and the category of an object surface. 


The angle presented on the display is not linearly related to the angle set 
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by the robot arm, and thus it can be seen that a-axis resolution and b-axis 
resolution of the optical mouse sensor are different. The characteristics of 
resolution vs. height can be used for height measurement and height variation 
of 0.2mm can be detected. Based on the feasibility of height measurement, 
the optical mouse sensor can be used for crack detection on a plane surface. 
An optical mouse sensor can feel the surface category in a non-contact way 
based on the characteristics of resolution vs. height of different surfaces. 

There are following advantages for the optical mouse sensor. The sensor 
interface is ready for use. The output of the system is digital, without the 
need of any additional analog-to-digital component. The driver and software 
for the sensing system is easily available. The outputs of the sensor can be 
intuitively displayed on the display and recorded on the hard disk, which 
greatly improves the convenience of the system. Because of the large-scale 
manufacture of the optical mouse sensor, the sensing system is cost-effective, 
and thus is easily available and generalized. 

In the future research, besides the non-contact tactile sensing proposed 
currently, based on the wavelength responsivity of an optical mouse sensor, a 
modified optical mouse sensor will be elementarily explored for surface color 
sensing. In daily life, such a home sensor can help the disabled, especially the 
blind. With the development of wireless technology in optical mouse sensors, 


such kind of application can be more flexible. 
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Chapter 5 


Combined Tactile Sensing with 


Optical Mouse Sensor 


5.1 Introduction 


This chapter introduces the concept of multifunctional sensing and considers 
combined sensing as one type of multifunctional sensing. Based the idea 
proposed in Chapter 4, using step-by-step table-look-up method, combined 


tactile sensing is proposed and verified on the sandpaper. 


5.2 Combined Sensing 


In this chapter, combined sensing can be consider as one type of multifunc- 
tional sensing. 
Depending on the functionality, the sensors can be divided into two 


groups, namely single function sensors and multifunctional sensors. As a 
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common sense, different sensors are used when there are different variables 
to be sensed. Different from the popular technique, known as the compound 
sensing, multifunctional sensors have been developed in the last decade in 
case the quantities being measured affect more than one single sensors input 
or compact structure of the sensing component is forcibly requested [40]. Em- 
ploying the required number of stand-alone sensors to realize multiple func- 
tions and measure multiple measurands is the conventional sensing method- 
ology. By a new concept, multifunctional sensors are quite different form 
integrated and compound sensors whereas the former one use the same struc- 
ture to realize multiple functions and measure several parameters, while the 
latter two employs individual sensor for each function and measurement. 
Additionally, the multifunctional sensing can improve the sensing object dis- 
criminative property of the sensor. In most of the cases, the proposed new 


methodology has the following advantages over the conventional method: 
1. Low cost 
2. Compactness 


3. Easy signal processing and interfacing 


5.3 Experiment 


5.3.1 Experiment Setup 


The diagram of the experiment system is shown in Fig. 5.1. A robot arm of 


model VS-6354DM is controlled by a robot controller of model RC5-VS6A. 
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Both of VS-6354DM and RC5-VS6A are manufactured by DENSO Corpo- 
ration. In each of X, Y and Z directions, the position repeatability of the 
robot arm of model VS-6354DM is + 0.02mm. 


Display 
: VS-6354DM 
Sensor 
RC5-VS6A S 
y Object Surface 


Figure 5.1: Diagram of experiment system for combined tactile sensing 


The optical mouse sensor in the experiment system is from an optical 
mouse manufactured by Logitech, and the product number is SOM-U20. The 
part number of the optical mouse sensor is S2599, which is manufactured by 
Agilent Technologies. The nominated resolution of S2599 is 400dpi. 


To ensure the isotropy of the object surfaces throughout the experiments, 
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sandpaper set MH913 of grit size #100, #320 and #1000 are selected as the 
object surfaces, which are manufactured by Musashi Holt Co., Ltd. 

Throughout the experiments, in the Windows XP operating system, the 
pointer speed is set to the fastest and the property ” Enhance pointer preci- 
sion” is enabled, which help improve the system sensitivity. The resolution 
of the display is set as 1024x768. 

During the experiment, when the optical mouse sensor is moved over the 
object surface by the robot arm, the pointer position on the display varies 
and the pointer position is recorded by a MATLAB application. 

The relative coordinate system is shown in Fig. 5.2. The x— y coordinate 
system is the coordinate system for the object surface and the a—b coordinate 
system is the coordinate system for the optical mouse sensor. a is the angle 
between a-axis and x-axis. 

Fig. 5.3 shows the height from lens reference plane to object surface, 


denoted as z. 


5.3.2 Initial Calibration 


Agilent Technologies provides a computer application named ” Cursor Tester” 
for the mouse bench test. Before the experiments, ” Cursor Tester” is em- 
ployed for direction calibration and parallelism calibration. The initial posi- 
tion and initial pose of the optical mouse sensor are adjusted by the robot 
arm while the outputs of ” Cursor Tester” are observed and analyzed. After 
several recurrences of calibration, when a is 0°, if the optical mouse sensor is 


moved in the z-axis direction, there is zero displacement in the vertical axis 
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be 


Figure 5.2: Sensing coordinate systems for the combined tactile sensing 


direction on the display; if the optical mouse sensor is moved in the y-axis 
direction, there is zero displacement in the horizontal axis direction on the 


display; and the lens reference plane is parallel to the object surface. 


12 


Lens 
Sensor 


i Object Surface ac Lens Reference Plane 


Figure 5.3: Height from lens reference plane to surface 


5.3.3 Experiment Procedures 


In the experiments, for a given z and a given a initially set by the robot 
arm, the optical mouse sensor is moved over the object surface at a uniform 
velocity of Imm/s and in the y-axis direction, while the pointer position 
on the display varies and the pointer position is recorded by the MATLAB 
application. 

In the MATLAB application, the sampling interval is set as 0.05s and 
each group of data includes 100 points, so the total sampling time for each 
group is 5s. The output of the sensing system is a sequence (2;,y;), which 
describes the track of the pointer on the display. x; is in the interval [1, 1024] 
and y; is in the interval [1, 768]. The length of the sequence is 100. 

In order to simplify data processing, the starting point coordinate of 
each pointer track curve is translated to (0, 0), because it is the relative 
displacement that makes sense. 

For sandpaper of grit size #100, #320 and #1000 respectively, z = [1.4, 
1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8] mm, and a = [0,10, 20, 30, 40, 50, 60, 70, 80, 


90)”. Finally, 240 sequences of dimension 100 x 2 are obtained. 
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5.4 Analysis 


The obtained data is organized as a table. The dimension of the data table 
is 3 x 8 x 10 x 100 x 2, that is 3 surface categories, 8 surface heights, 10 
surface translation directions and 100 pairs of (xj, yi). 

Based on the relativity of movement, the translation direction and the 
height of the object surface can be obtained from a and z respectively. The 


measurement of œ is independent of z. 


5.4.1 Surface Translation Direction Measurement 


The 1st dimension and 2nd dimension of the data table, that is (x,,y;), 
are employed to calculate the angle a. One estimate of tan(a) is the so- 
lution in the least squares sense to the overdetermined system of equations 
Yioox1 tanta) = X100x<1. For calculation of each tan(a’), the average of 24 
solutions of equations corresponding to 3 surface categories and 8 values of z 
are used. The relation between the estimated angle a’ and the actual angle 
a is shown in Fig. 5.4. 

It can be seen that a’ and a is not linearly related and a cubic polynomial 
provides a good fitting. That is, the angle a’ presented on the display is not 


linearly related to the angle a set by the robot arm. 


5.4.2 Surface Category Discrimination 


As a human hand touches the sandpaper of different grit size, the feel is 
slightly different, and an optical mouse sensor can also feel the difference 


between the sandpapers with different grit size in a non-contact way. 
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Figure 5.4: Estimated angle vs. actual angle for the surface translation 
direction measurement 


The resolution vs. height characteristics of the 3 different surfaces are 
shown in Fig. 5.5. For each object surface, the characteristics of resolution 
vs. height are specific, and a specific characteristics curve can be recognized 
by a specific curve segment. In order to recognize a specific characteristics 
curve, another additional optical mouse sensor cooperates, with a calibrated 
height difference (e.g. 0.2mm) between the two optical mouse sensors. 

After the translation direction measurement, for a given a (e.g. a = 0°), 
based on the Ist dimension, the 2nd dimension and 4th dimension of the 
data table, by matching the outputs of the two optical mouse sensors, a 
specific characteristics curve can be identified, and therefore the category of 


the object surface can be discriminated. 
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Figure 5.5: Resolution vs. height characteristics for surface category discrim- 
ination 


5.4.3 Surface Height Measurement 


For different object surface, the position and range of the monotonic interval 
of the characteristics of resolution vs. height are different. In the characteris- 
tics of resolution vs. height of an object surface, if the height variation range 
of the object surface is less than the range of a monotonic interval of the 
characteristics of resolution vs. height, by setting the initial height between 
the lens reference plane and the object surface properly, the height variation 
interval of the object surface can fall into the monotonic interval. In such a 
condition, the characteristics of resolution vs. height can be used for height 
measurement effectively. Besides, the position and range of the monotonic 
interval of the characteristics of resolution vs. height can also be adjusted 
by tuning the focus of the lens. 


It can be seen that height variation of 0.2mm can be detected. For dif- 
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ferent object surface, the position and range of the monotonic interval of the 
characteristics of resolution vs. height are different. 

After the surface category discrimination, for a given angle (e.g. a = 0°) 
and a given surface category (e.g. Sandpaper #320), the height of the optical 
mouse sensor can be measured, as shown in Fig. 5.6. Cubic spline provides 


a good fitting. 


-O-Original curve 
— Cubic spline fitted curve 
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Figure 5.6: Surface height measurement in the combined tactile sensing 


5.4.4 Summary 


When an object surface is moved underneath the optical mouse sensor with 
the output of the optical mouse sensor recorded, the translation direction, 
the category and the height of the object surface can be obtained with a step- 
by-step table-look-up method based on a predefined data table, as shown in 
Fig. 5.7. Firstly, with a recorded pointer track and an interpolation, œ can be 


obtained based on Fig. 5.4; secondly, with a, the displacement recorded by 


T7 


the pointer track is transformed to the the displacement corresponding to a 
specific translation direction (e.g. œ = 0°), and then the surface category can 
be discriminated with a double table-look-up based on Fig. 5.5; and thirdly, 
with the specific translation direction (e.g. œ = 0°) and the surface category, 
z can be obtained with a table-look-up and an interpolation based on Fig. 


5.6. The predefined table can be extended to help improve the combined 


tactile sensing. 


8 10 100 


Surface 
Category 


Figure 5.7: Diagram of step-by-step sensing procedures in combined tactile 
sensing 


5.5 Discussion 


The angle presented on the display is not linearly related to the angle set by 
the robot arm. It can be concluded that the a-axis resolution is less than the 
b-axis resolution. This is because the illumination of the LED is from the 


side, not vertically and symmetrically. The default resolution is referred to 
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the b-axis resolution. 

In the experiments above, x is in the interval [1, 1024] and y is in the 
interval [1, 768]. In actual application, there may be some cases when x and 
y are out of ranges above. In fact, this is a limitation caused by the driver 
software of the computer mouse. In actual application, the output signals 
of the optical mouse sensor, which is a two-channel quadrature output, can 
be directly used for processing. In such a way, without the intervention of 
the operating system and the driver software, smaller height variation can be 
measured and the measurement speed can be improved. Furthermore, the 
two-channel quadrature output can be transformed by logic circuits to be 
used for PWM (Pulse Width Modulation) control in some industry applica- 
tions. 

Besides the height from lens reference plane to object surface and the 
category of the object surface, many other factors, such as the velocity of 
the sensor, LED current level and the wavelength responsibility of the LED, 
influence the resolution of the optical mouse sensor and the output of the 
optical mouse sensor. In our current research, these parameters are constant, 
and the effects of them will be studied in further research work. 

Furthermore, the essence of an optical mouse sensor is optical flow [14]. 
For an optical mouse, the lens focuses the camera onto the surface directly 
below the mouse and thus an optical mouse only tracks the surface directly 
touching or very slightly above the surface it is tracking. By interfacing an 
optical mouse sensor to a microcontroller and replacing the default lens with 
a lens focusing at a distance away, 1t should be possible to harness this cheap 


optical flow technology for many more tasks. 
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5.6 Conclusion 


Instead of as a traditional position sensor, an optical mouse sensor can be 
used for combined tactile sensing to feel the translation direction, the cate- 
gory and the height of an object surface in a step-by-step way. 

There are following advantages for the optical mouse sensor. The sensor 
interface is ready for use. The output of the system is digital, without the 
need of any additional analog-to-digital component. The driver and software 
for the sensing system is easily available. The outputs of the sensor can be 
intuitively displayed on the display and recorded on the hard disk, which 
greatly improves the convenience of the system. Because of the large-scale 
manufacture of the optical mouse sensor, the sensing system is cost-effective, 
and thus is easily available and generalized. 

In the future research, besides the combined tactile sensing proposed cur- 
rently, based on the wavelength responsivity of an optical mouse sensor, a 
modified optical mouse sensor will be elementarily explored for surface color 
sensing. In daily life, such a home sensor can help the disabled, especially the 
blind. With the development of wireless technology in optical mouse sensors, 


such kind of application can be more flexible. 
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Chapter 6 


Conclusions and 


Recommendations 


6.1 Introduction 


The overall goal of the thesis is to apply the optical mouse sensor to the field 
of tactile sensing. The optical mouse sensor is used for surface shape analyz- 
ing, non-contact tactile sensing and combined tactile sensing respectively. 

In Chapter 3, a device for analyzing the surface shape is reported, tak- 
ing advantage of an optical mouse sensor. For smooth surface, the device 
can record the surface shape digitally, and for rough surface, the device can 
estimate the roughness in a statistical way. The resolution of the device is 
restricted by the resolution of the optical mouse sensor and the size of the 
ballpoint in the ” antenna”. 

In Chapter 4, two series of experiments are performed on rubber surfaces 


and whitepaper. For the rubber surface, the periodically-square-shaped sur- 
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face and the periodically-arc-shaped surface can be distinguished and some 
shape parameters of the two object surfaces can be obtained. As a sensor 
working in motion, the translation direction of the object surfaces can be 
obtained. For the whitepaper, The translation direction, the height and the 
category of an object surface can be sensed. 

In Chapter 5, based on the experimental characteristics of the optical 
mouse sensor, with a step-by-step table-look-up method, a combined tactile 
sensing system is proposed. When an object surface is moved underneath the 
optical mouse sensor, the translation direction, the category and the height 
of the object surface can be obtained with the recorded output of the optical 


mouse sensor based on a predefined data table. 


6.2 Recommendations for Future Work 


Besides the distance from lens reference plane to surface, many other fac- 
tors, such as the velocity of the sensor, LED current level, the wavelength 
responsibility of the LED and the material of the object surface, influence the 
resolution of the optical mouse sensor and the output of the optical mouse 
sensor. In our current research, these parameters are constant, and the effects 
of them can be studied in further research work. 

Besides the LED-based optical mouse sensor, the laser optical mouse 
sensor with superior performance is gradually popularized. The laser opti- 
cal mouse sensor is enabled with Agilent's LaserStream Technology. Laser- 
Stream navigation engines provide much finer precision and more accurate 


tracking on a wider variety of surfaces, part of which proves difficult for 
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traditional LED-based optical navigation, such as glossy and reflective sur- 
faces. The adoption of the LaserStream Technology will help improve the 
performance and application of the proposed system. 

Currently, the optical mouse sensors have been used in the robot odome- 
try. Not just for odometry, we expect that other useful surface configuration 
information can be obtained and transferred to the robot by the optical mouse 
sensor. In the future research, besides surface height variation sensing, based 
on the wavelength responsivity of an optical mouse sensor, a modified optical 
mouse sensor will be elementarily explored for surface color sensing. In daily 
life, such a home sensor can help the disabled, especially the blind. With 
the development of wireless technology in optical mouse sensors, such kind 
of application can be more flexible. 

Furthermore, the essence of an optical mouse sensor is optical flow. For an 
optical mouse, the lens focuses the camera onto the surface directly below the 
mouse and thus an optical mouse only tracks the surface directly touching or 
very slightly above the surface it is tracking. By interfacing an optical mouse 
sensor to a microcontroller and replacing the default lens with a lens focusing 
at a distance away, it should be possible to harness this cheap optical flow 


technology for many more tasks. 
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